Inverse rendering of visual material properties

ABSTRACT

Techniques described herein are directed to a system and methods for generating 3D models of an object which accurately depict reflective properties of the object. To do this, an image of an object is captured and a rendered image of the object is generated from the image. The system then generates a lighting effect which approximates an effect of the actual light source on the appearance of the object when the image was captured. A number of rendered images of the object are generated using the lighting effect, each having different visual material property values. Once the rendered images have been generated, the system may compare the generated rendered images to the actual image in order to identify the rendered image which best approximates the actual image. The visual material property values associated with the best approximation are then assigned to the object.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/823,055, filed Nov. 27, 2017, the entire contents of which are herebyincorporated by reference in its entirety for all purposes.

BACKGROUND

In a number of industries, three-dimensional (3D) models may be used torepresent various objects. For example, any industry that uses or sellsobjects may need to understand how different objects interact or howthey might appear together. The generation of such a 3D model mayrequire that a system obtain an image of the surface of an object, whichmay be rendered or displayed as a two-dimensional image via 3D renderingor displayed as a three-dimensional image.

Various techniques exist for generating 3D models from objects. Forexample, a given object may be scanned from a number of differentangles, and the scanned images can then be combined to generate the 3Dimage of the object. In another example, a 3D model may be createdmanually by an artist or drafter.

Often, when an object is being scanned, it is desirable to show nearexact color and visual material properties to more accurately depict howthe object looks in the real world. However, due to lighting andreflectiveness of a sample being scanned, some visual materialproperties, such as whether an item is metallic or has rough surfaces,can be hard to assess. For example, it is often difficult to assessvisual material properties such as metalness or roughness. “Metalness”is a measure of how metallic something is. “Roughness” is a measure ofhow shiny or reflective something is. If something is very rough, it isnot shiny at all. If something is not rough at all, then it has verysharp highlights (e.g., shiny).

In some conventional systems, default visual material properties areapplied to every object, resulting in generated 3D models that lookequally shiny. The resulting 3D models lack realism. In some otherconventional systems, an artist may estimate various visual materialproperties for an object based on the captured image. However, thisoften results in inconsistencies and is error prone in general. Someconventional systems may use a device, such as a Gonioreflectometer,which directly captures the bidirectional reflectance (BRDF) of amaterial per pixel. Devices such as this take thousands of images of anobject from every possible camera angle, and from every possiblelighting direction. However, capturing the number of images of an objectrequired by the device can be inefficient and costly.

Embodiments of the invention address these and other problems,individually and collectively.

BRIEF SUMMARY

Techniques described herein are directed to a system and methods forgenerating a 3D model of an object which is accurately depicted withrespect to visual material properties. In some embodiments, a number ofimages of an object are captured in order to generate a virtual image ofthe object. The system then generates a virtual light source whichapproximates an actual light source with which the images were captured.The virtual light source may be used to generate a number of virtualimages of the object, each having different visual material propertyvalues. Once generated, the system may compare the generated virtualimages to the actual image in order to identify the virtual image whichbest approximates the actual image. The approximation may be calculatedas a delta or image error. Once the lowest image error has beenidentified for the virtual images, the visual material property valuesassociated with the virtual image having the lowest image error areassigned to the object. In some embodiments, the assigned visualmaterial property values may be refined by repeating the process using asmaller range of visual material property values. In these embodiments,the smaller range of visual material property values may be set toinclude the assigned visual material property values.

One embodiment of the disclosure is directed to a method of generating athree-dimensional (3D) model, comprising receiving a captured image ofan object using a light source, identifying a section of the objectbased on the captured image, generating a virtual light source thatapproximates the light source. The method further comprises for the atleast a first section of the object, performing operations thatcomprise: generating a grid of virtual images, using the generatedvirtual light source, with each virtual image having a specified visualmaterial property value according to a position within the grid ofvirtual images array, comparing an individual virtual image in the gridof virtual images to the captured image, calculating an image errorbased on the comparison between the individual virtual image and thecaptured image, and assigning a visual material property valueassociated with the individual virtual image to the section based uponthe image error.

Another embodiment of the disclosure is directed to a 3D imaging systemcomprising one or more camera devices, a processor, and a memoryincluding instructions that, when executed with the processor, cause thesystem to, at least: generate a virtual light source that approximates alight source, receive one or more captured images of an object with thelight source, receive user information regarding one or more operationalconstraints, generate a grid of virtual images, using the generatedvirtual light source, with each virtual image having a specified visualmaterial property value according to a position within the grid, compareat least one virtual image in the grid to the one or more capturedimages, calculate, based on the comparison between each virtual image inthe grid and the one or more captured images, an image error, identify avirtual image of the grid of virtual images associated with a minimumimage error, and assign at least one visual material property valueassociated with the identified virtual image to the first section.

Yet another embodiment of the disclosure is directed to a 3D imagingapparatus comprising one or more optical sensors configured to capturean image of an object, a light source, and a control unitcommunicatively coupled to the one or more optical sensors, the controlunit configured to: receive one or more captured images of an objectwith the light source, generate a virtual light source that approximatesthe light source; generate a grid of virtual images, using the generatedvirtual light source, with each virtual image having a specified visualmaterial property value according to a position within the grid, compareat least one virtual image in the grid to the one or more capturedimages, calculate, based on the comparison between each virtual image inthe grid and the one or more captured images, an image error, identify avirtual image of the grid of virtual images associated with an imageerror, and assign at least one visual material property value associatedwith the identified virtual image to the first section.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments in accordance with the present disclosure will bedescribed with reference to the drawings, in which:

FIG. 1 depicts an illustrative example of a 3D imaging system that maybe implemented in accordance with at least some embodiments;

FIG. 2 depicts a system architecture for a 3D imaging system that may beconfigured to generate a 3D model of an object using inverse renderingof visual material properties in accordance with at least someembodiments;

FIG. 3 depicts a flow chart illustrating a process for that utilizesinverse rendering of material in accordance with at least someembodiments;

FIG. 4 depicts a flow chart illustrating a process for capturing imagesin generating a 3D model in accordance with at least some embodiments;

FIG. 5 depicts an illustrative example of information that may be usedto generate a virtual light source in accordance with at least someembodiments;

FIG. 6 depicts an illustrative example of a grid of virtual images thatmay be generated in accordance with at least some embodiments;

FIG. 7 depicts an illustrative example of a process for comparingvirtual images to an actual image and approximating visual materialproperty values based on that comparison in accordance with at leastsome embodiments;

FIG. 8 depicts an illustrative example of a process for refining visualmaterial property values by adjusting a range of visual materialproperty values and generating a new grid using the adjusted range ofvisual material property values in accordance with at least someembodiments; and

FIG. 9 depicts one potential implementation of the system described inaccordance with at least some embodiments.

DETAILED DESCRIPTION

In the following description, various embodiments will be described. Forpurposes of explanation, specific configurations and details are setforth in order to provide a thorough understanding of the embodiments.However, it will also be apparent to one skilled in the art that theembodiments may be practiced without the specific details. Furthermore,well-known features may be omitted or simplified in order not to obscurethe embodiment being described.

Techniques described herein are directed to a three-dimensional (3D)imaging system configured to generate a 3D model of an object usinginverse rendering of material properties. The imaging system may iteratethrough a number of sections of an object when generating a 3D model ofthe object. In some embodiments, a range of material properties may bepre-defined and stored in the imaging system. Once an appropriate rangeof material properties is selected, the generation of the 3D model mayoccur in accordance with this range information. For example, theimaging system may store an upper bound and/or lower bound informationfor specular values associated with different types of materials. Upondetecting that a section of an object likely consists of a materialtype, the imaging system may retrieve and apply the upper and/or lowerbound values of the material properties.

In some embodiments, the 3D imaging system may include a turntable,where an object may be staged and rotated. In these embodiments, theobject may be rotated and photos may be taken from multiple differentangles. In other embodiments, a full array of cameras can be usedwithout a turntable. The difference between a turntable system and afull array is that a turntable simulates the presence of hundreds ofcameras, whereas a full array may literally be composed of hundreds ofcameras. In a full array system, there is no turntable, and cameras aretypically arranged facing inward to scan the object in the center. Onereason a person might want to use a full array system as opposed to aturntable system is to scan objects that have a tendency to move duringscanning (e.g., people). A full array permits scanning of such an objectalmost instantaneously, avoiding issues with movement of the object. Asused herein, an imaging system is any system capable of capturing imageinformation for an object in order to generate a 3D model, with orwithout a turntable.

Physically Based Rendering (PBR) is the process in which a 3D modelrenderer seeks to render graphics to more accurately model the flow oflight in the real world. Often, PBR is directed to the accurate orrealistic representation of visual material properties for an object.Visual material properties may include any property which affects theappearance of an object, such as intensity, gloss, metallicity (alsoreferred to as metalness), roughness (e.g., a visual effect resultingfrom microsurfaces), or any other suitable visual property of amaterial.

In some embodiments, visual material properties may be specific to aparticular shading model. For example, different shading models may usedifferent visual material properties when rendering 3D models. It shouldbe noted that the techniques described herein may be applicable usingany suitable visual material properties, and not just thoseillustratively used herein.

FIG. 1 depicts an illustrative example of a 3D imaging system that maybe implemented in accordance with at least some embodiments. In FIG. 1,one or more camera devices 102 may be configured to obtain a set ofimages of an object 104. In some embodiments, the 3D imaging system mayalso include a staging object 106 placed within a scene associated withthe object 104. The object 104 may be positioned on a staging platform108. A light source 110 may illuminate the object 104 and thesurrounding area.

For clarity, a certain number of components are shown in FIG. 1. It isunderstood, however, that embodiments of the invention may include morethan one of each component. In addition, some embodiments of theinvention may include fewer than or greater than all of the componentsshown in FIG. 1. In addition, the components in FIG. 1 may communicatevia any suitable communication medium (including the Internet), usingany suitable communication protocol.

In some embodiments, the 3D imaging system may include one or morecamera devices 102 configured to capture a one or more images of theobject 104. In some embodiments, the one or more camera devices 102 mayinclude an array of camera devices. In these embodiments, each cameradevice in the array of camera devices may be configured to capture animage of a different section of a surface of the object 104. In someembodiments, the one or more camera devices 102 may consist of a singlecamera device. The camera devices 102 may each be capable of exhibitinga range of movement 112.

In some embodiments, the object 104 may include a number of separatesections or regions. Different sections of an object 104 may beidentified to correspond to differences in materials for thoserespective sections. For example, an object 104 may be made up of ametallic section and a plastic section. In this example, each of themetallic and plastic sections would be identified as separate sectionsof the object 104. To identify various sections of the object 104, the3D imaging system may identify variances in material properties of theobject 104 over its surface. For example, a material's appearance maychange over its surface as a result of a change in color, texture,roughness, or metalness values. Thus, separate sections of the object104 may be identified as being associated with different materialsbecause those separate sections have different specular values (e.g.,roughness, or metalness values). An object 104 may include any number ofsections with any number of different types of materials. For example, atelevision remote control may have two sections, which consist of twodifferent types of materials: one for the body of the remote, andanother for the buttons of the remote control. In some embodiments, the3D imaging system may begin one or more of the techniques describedherein by identifying a number of sections of the object.

In some embodiments, the 3D imaging system may use a staging object 106to approximate the existing light source 110. A staging object 106 maybe any object having a known geometry and specular values. For example,the staging object 106 may be a chrome ball which has a maximummetalness value and a minimal roughness value. The staging object 106may be used to approximate a position and intensity of the light source110 relative to the scene.

For example, the 3D imaging system may calculate the position of thelight source 110 based on a position of a highlight 114 identified onthe staging object 106, which may indicate an angle of reflection. Basedon the determined position of the light source 110, the 3D imagingsystem may approximate a virtual light source to be used in thetechniques described herein. In some embodiments, the light source 110may be in a known location and have a known intensity. In theseembodiments, it may be unnecessary for the 3D imaging system to includea staging object 106.

In some embodiments, the object 104 may be placed on a staging platform108. The staging platform 108 may be capable of being rotated orotherwise moved. For example, the staging platform 108 may be fittedwith a motor or other actuator to be used in angular rotation of theplatform. During the course of obtaining a set of images, a control unit(not shown) may cause the staging platform 108 to reposition the objecta number of times so that sets of images may be taken of the object 104at each position.

In accordance with some embodiments, the 3D imaging system illustratedin FIG. 1 may be used to accurately depict an object 104 using a virtualobject having visual material properties approximate to the object 104.In these embodiments, the 3D imaging system may generate a virtual lightsource (using a process described in greater detail below). The 3Dimaging system may then identify a section of the object for whichvisual material properties are to be approximated. Using the virtuallight source, the 3D imaging system may generate a grid of virtualimages having predetermined visual material property values. The gridmay include two or more dimensions, each of which represent mappings ofvisual material property combinations. Predetermined visual materialproperty values may be maintained by the system or provided by a user.By way of illustration, a user may provide a range of values for avisual material property, such as metalness, as one operationalconstraint for the grid. In this illustration, the visual materialproperty range provided for the grid may include a first visual materialproperty range for “metalness” from 0 through 9 units. One advantage topicking a particular specular range is optimization, in that rather thancreate a grid for every possible range of units that a visual materialproperty can consist of, the grid includes only those values of thevisual material property likely to be relevant. A second visual materialproperty, such as roughness, may be provided and the two visual materialproperty value sets can be arranged in a grid representing all possiblecombinations of the two visual material properties (i.e., (1, 1), (1,2), (1, 3) . . . (2, 1), (2, 2), (2, 3) . . . (9, 7), (9, 8), (9, 9)).In some embodiments, the constraints for the grid may be defined by auser. It should be noted that although increments are depicted in theabove example as being in integers, the grid may include any suitableincrements. The 3D imaging system may then generate a number of virtualimages to fill the generated grid of virtual images. Each of thegenerated virtual images may be generated for its corresponding gridcoordinates using the visual material properties assigned to thosecoordinates.

Once virtual images have been generated for each of the fields in thegrid of virtual images, each of those images may be compared to anactual image of the object 104 obtained. The 3D imaging system may thendetermine a delta between the actual image and each of the virtualimages. In some embodiments, the delta may be represented as a level oferror (e.g., a mean squared error) or the delta may be represented as anabsolute difference. In some embodiments, these deltas may be maintainedin a grid format in which each delta is stored so that it correspondswith its associated virtual image. Once the deltas have been identifiedfor each of the virtual images, the 3D imaging system may identify theminimum delta from those determined. The 3D imaging system may thenassign the visual material properties assigned to the associated virtualimage to the section of the object.

The process described above may be repeated a number of times for eachidentified section of an object 104. For example, if the 3D imagingsystem determines that the object has three sections, then the 3Dimaging system may generate a grid for each of the three sections usingthe same virtual light source. The 3D imaging system may then assignvisual material properties to each of the sections of the object 104using the described process. In some embodiments, the visual materialproperties of a section may be refined by narrowing the range of visualmaterial property values based on those identified in the above processand repeating the process using that narrower range of values.

In some embodiments, the one or more camera devices 102 may beconfigured to be rotated or moved. For example, the cameras may bepositioned on moveable tracks or rails in order to cause a camera, a rowof cameras, and/or a column of cameras to be repositioned in accordancewith instructions provided by a control unit. In some embodiments, oneor more cameras may be installed on a mechanical arm assembly. In someembodiments, each of the one or more camera devices 102 may beconfigured such that a level of pan or tilt for that camera may beadjusted.

FIG. 2 depicts a system architecture for a 3D imaging system that may beconfigured to generate a 3D model of an object using inverse renderingof visual material properties in accordance with at least someembodiments. In FIG. 2, a control unit may be in communication with anumber of other components, including at least a staging device 204, asensor array 206, and an interface device 208.

The control unit 202 may be any type of computing device configured tocapture 3D images from an object. In some embodiments, the control unit202 may be executed by one or more virtual machines implemented in ahosted computing environment. The hosted computing environment mayinclude one or more rapidly provisioned and released computingresources, which computing resources may include computing, networking,and/or storage devices. A hosted computing environment may also bereferred to as a cloud-computing environment.

In one illustrative configuration, the control unit 202 may include atleast one memory 210 and one or more processing units (or processor(s))212. The processor(s) 212 may be implemented as appropriate in hardware,computer-executable instructions, firmware or combinations thereof.Computer-executable instruction or firmware implementations of theprocessor(s) 212 may include computer-executable or machine executableinstructions written in any suitable programming language to perform thevarious functions described.

The memory 210 may store program instructions that are loadable andexecutable on the processor(s) 212, as well as data generated during theexecution of these programs. Depending on the configuration and type ofcontrol unit 202, the memory 210 may be volatile (such as random accessmemory (RAM)) and/or non-volatile (such as read-only memory (ROM), flashmemory, etc.). The control unit 202 may also include additional storage214, such as either removable storage or non-removable storageincluding, but not limited to, magnetic storage, optical disks, and/ortape storage. The disk drives and their associated computer-readablemedia may provide non-volatile storage of computer-readableinstructions, data structures, program modules, and other data for thecomputing devices. In some implementations, the memory 210 may includemultiple different types of memory, such as static random access memory(SRAM), dynamic random access memory (DRAM) or ROM. Turning to thecontents of the memory 210 in more detail, the memory 210 may include anoperating system 216 and one or more application programs or servicesfor implementing the features disclosed herein including at least amodule for generating virtual images of an object or section of anobject (virtual imaging module 218) and/or a module for comparingvirtual images to an actual image to determine visual materialproperties of the actual image (image comparison module 220). The memory210 may also include imaging and lighting data 222, which providesinformation used by the 3D imaging system. In some embodiments, theimaging and lighting data 222 may be stored in a database.

The memory 210 and the additional storage 214, both removable andnon-removable, are examples of computer-readable storage media. Forexample, computer-readable storage media may include volatile ornon-volatile, removable or non-removable media implemented in any methodor technology for storage of information such as computer-readableinstructions, data structures, program modules or other data. As usedherein, modules may refer to programming modules executed by computingsystems (e.g., processors) that are installed on and/or executed fromthe control unit 202. The control unit 202 may also containcommunications connection(s) 224 that allow the control unit 202 tocommunicate with a stored database, another computing device or server,user terminals, and/or other components of the imaging system. Thecontrol unit 202 may also include input/output (I/O) device(s) and/orports 226, such as for enabling connection with a keyboard, a mouse, apen, a voice input device, a touch input device, a display, speakers, aprinter, etc.

Turning to the contents of the memory 210 in more detail, the memory 210may include an operating system 216, a database containing imaging andlighting data 222 and the one or more application programs or servicesfor implementing the features disclosed herein, including a virtualimaging module 218 and an image comparison module 220.

In some embodiments, the virtual imaging module 218 may be configuredto, in conjunction with the processors 212, generate a virtual lightsource using information obtained from an image and generate a number ofvirtual images of an object, each with predetermined visual materialproperties using the virtual light source. In accordance with at leastsome embodiments, the virtual imaging module 218 may identify lightinginformation in an image by measuring the light indicated on a stagingobject, such as a chrome (or mirrored) ball. Based on this information,the virtual imaging module 218 may identify a location, color, andintensity of a light source. The virtual imaging module 218 may thengenerate a virtual light source that approximates the detected light. Inthis way, light reflected off of the staging object may be detected andused by the virtual imaging module 218 to generate a virtualapproximation of an actual light source.

Once a virtual light source has been generated to approximate the actuallight source, the virtual imaging module 218 may generate a number ofvirtual images using that light source. In some embodiments, the virtualimages may be arranged in, or associated with, grid coordinates. Inthese embodiments, the columns and rows of the grid may representparticular visual material property values such that the virtual imageassociated with a particular data field is generated using those visualmaterial property values. When generating the virtual images, thevirtual imaging module 218 estimate a location of a highlight for thesection of the object within the virtual image based on the geometry andpose (location and orientation) of the section of the object as well asthe position of the virtual light source. The virtual imaging module 218may then generate a highlight in the estimated location using the visualmaterial properties assigned to a grid coordinate.

In some embodiments, the image comparison module 220 may be configuredto, in conjunction with the processors 212, compare each virtual imagein a grid of virtual images to an actual image of an object, calculate adelta between each of the virtual images in the grid and the actualimage, and approximate visual material properties for an object byidentifying a minimum delta. In some embodiments, the image comparisonmodule 220 may compare only a segment of the virtual image in which ahighlight is estimated to exist to a corresponding segment in the actualimage. In some embodiments, the image comparison module 220 maydetermine (e.g., based on a number of relatively high deltas), that ahighlight on the generated virtual image has been placed in the wronglocation. The image comparison module 220 may then identify a correctlocation for the highlight and may call an instance of the virtualimaging module 218 to regenerate the grid of virtual images using thecorrected highlight location. Comparison of a virtual image to an actualimage may be performed in a number of ways. In some embodiments, theimage comparison module 220 may make a pixel-by-pixel comparison inwhich pixel values (e.g., red, green, and blue values) are compared foreach corresponding pixel in the two images.

A delta may then be calculated as a function of the difference in valuesidentified. In some embodiments, the delta may be represented by anabsolute difference or mean-squared error. In some embodiments, deltasmay be calculated and stored in relation to each of the virtual imagesin the grid of virtual images. In at least some of these embodiments,these deltas may be stored in a grid of deltas.

In some embodiments, a staging device 204 may be any device or structureconfigured to manipulate a position of an item for the purpose ofobtaining image information. Staging platform 106 described in FIG. 1may be an example staging device 204. In some embodiments, the stagingdevice 204 may include an object positioning platform 226 and a platformrotator 228. The object positioning platform 226 may be a rotatableplatform upon which one or more items can be placed for imaging of theitems using the sensor array 206. In some embodiments, the rotatableplatform may be configured to be rotated by the platform rotator 228 inorder to reposition the item on the platform. The platform rotator 228may be a motor configured to, upon receiving instructions from thecontrol unit 202, activate and cause the object positioning platform torotate.

In some embodiments, the sensor array 206 may include a number of sensordevices 230 and one or more sensor device adjustment mechanisms 232. Thesensor devices 230 may include a number of camera devices, one or moreof which may be a range camera device (e.g., a depth sensor) capable ofgenerating a range image, and a number of which may be camerasconfigured to capture image information. The one or more camera devices102 depicted in FIG. 1 are an example of a sensor array 206 that may beimplemented. In some embodiments, the sensor array 206 may include anumber of camera devices arranged in an array or matrix (e.g., in rowsand columns). Each of the camera devices in the array of camera devicesmay be positioned (e.g., having a tilt and position) to capture imageinformation with respect to a particular respective field of view. Insome embodiments, each of the camera devices may be configured to berepositioned in order to alter that camera device's tilt, pan, and/orlevel of magnification to capture an image of a specified field of view.

The sensor device adjustment mechanism 232 may be any mechanism havingmeans for adjusting a pan, tilt, and/or position of one or more sensordevices. In some embodiments, the sensor device adjustment mechanism 232may be a mechanical or robotic arm. In some embodiments, the one or moresensor devices may be arranged on a track or rail, in which case thesensor device adjustment mechanism 232 may include a means for movingthe sensor device along the track or rail. In some embodiments, the oneor more sensor devices may be arranged on a tripod or pole, in whichcase the sensor device adjustment mechanism 232 may include a means forproviding angular rotation for the sensor devices. One skilled in theart, after considering the teachings of this disclosure, would easilyrecognize a number of sensor device adjustment mechanisms that may beemployed with the disclosed system.

In some embodiments, the 3D imaging system may include an interfacedevice 208. An interface device may include any combination of displaydevice 234 and/or input device 236. In some embodiments, each of thedisplay device 234 and the input device 236 may be separate deviceswhich are communicatively coupled. The display device may include anymeans for presenting information to a user or users. In someembodiments, the display device may include outputs for audiopresentation of data. In some embodiments, the display device may bewearable. For example, the display device 234 may be a virtual reality(VR) or augmented reality (AR) headset.

The input devices 236 may be any device configured to receive input froma user and convert that input into machine executable instructions. Insome embodiments, the input devices 236 may include mouse, keyboards,joysticks, or any other suitable means of receiving user input. In someembodiments, the interface device 208 may be used, in conjunction withthe control unit 202, to manipulate the sensor array 206 and/orcomponents of the staging device 204.

FIG. 3 depicts a flow chart illustrating a process for generating a 3Dmodel using inverse rendering of visual material properties inaccordance with at least some embodiments. Some or all of the process300 (or any other processes described herein, or variations, and/orcombinations thereof) may be performed under the control of a computersystem configured with executable instructions and may be implemented ascode (e.g., executable instructions, one or more computer programs, orone or more applications) executing collectively on one or moreprocessors, by hardware or combinations thereof. The code may be storedon a computer-readable storage medium, for example, in the form of acomputer program comprising a plurality of instructions executable byone or more processors. The computer-readable storage medium may benon-transitory. In some embodiments, the computer system that performsthe process 300 may be a 3D imaging system that includes some or all ofthe components depicted in FIG. 2.

Process 300 may begin at 302, when a request is received by a computersystem to generate a 3D image of an object. In some embodiments, theobject may be positioned within a staging area (e.g., on an objectpositioning platform 226) in view of an array of camera devices. In someembodiments, the computer system may provide an interface (e.g., via theinterface device 208 of FIG. 2), to the user. At the interface, a usermay request to capture 3D imaging information for an object. In someembodiments, the request may include an indication of a material for anobject and/or a range of visual material property values. For example, auser may identify two or more distinct materials for an object and/or aspecific range of visual material properties that should be used in theprocess 300.

There may be a number of reasons for a user to provide a range of visualmaterial property values. For example, a user may know a particularrange for the given object that is likely a close match. In this sameexample, a user may select a particular range in order to achieve agreater efficiency of generating a 3D model. As can be envisioned, usinga smaller visual material property range will result in achieving agreater level of efficiency in generating a 3D model. However, using asmaller visual material property range will also result in potentialloss of accuracy (e.g., there may be a configuration of visual materialproperties that had a lower error than the limited range). Accordingly,a user may wish to configure a range of visual material property valuesbased on whether greater accuracy is needed.

At 304, the computer system captures one or more images of an object(e.g., the object 104 depicted in FIG. 1). The system may capture theone or more images using the process shown in the flow diagram of FIG.4. Several properties of the object may be identified during the imagingprocess. For example, the system may recognize the geometry of theobject, the lighting condition, the diffuse/color texture of the object,and material properties of the object. In some embodiments, High DynamicRange Imaging (HDRI) techniques are employed. HDRI techniques allow forthe capture of greater details and a dynamic range of luminosity for theobject.

At 306, the computer system may identify separate sections within anobject. In some embodiments, this may involve the identification ofvariances in properties of the object detected over its surface. In someembodiments, the computer system may provide an interface (e.g., via theinterface device 208 of FIG. 2), to the user, which may enable the userto provide an indication of different sections of the object as well aspotential materials for those sections. For example, in someembodiments, an indication may be received from a user that an objectincludes three sections that consist of a top section, a bottom section,and a middle section. In this example, the user may also indicate a typeof material for each of the sections. In some embodiments, the 3Dimaging system may identify each of the sections of the objectautomatically (e.g., without user interaction) by identifying variancesin material properties. For example, the 3D imaging system may detect achange in coloring or a change in reflectance values for the object. Inthis example, the area associated with a first set of materialproperties may be associated with a first section of the object and anarea associated with a second set of material properties may beassociated with a second section of the object.

In some embodiments, the computer system may also use machine visionand/or one or more machine learning (ML) techniques to identify theseparate sections of the object. For example, the computer system mayidentify, using ML, that the object is a hammer. In this example, thecomputer system may determine that a hammer typically includes a headsection and a shaft section. The computer system may also determine thatthe head section is typically made of a metallic material and the shaftsection is typically made of a wood or plastic material. In someembodiments, the staging area in which the object is located may bemarked with one or more identifiers or patterns that may be used, whencaptured within image information, to identify different sections of anobject. For example, the computer system may attempt to locate one ormore markers within an image captured of the object and, upon detectingthe one or more markers, may determine separate sections of the objectbased on the location of features of the object in relation to themarkers.

At 307, the computer system may generate a virtual light source based onlighting conditions detected in a captured image. In some embodiments,the computer system may detect a staging object within the image thatincludes a known geometry and/or visual material property values. Fromthe location and intensity of a highlight resulting from a reflection ofthe light source on the staging object, the computer system maydetermine an approximate location of the light source. In someembodiments, the location and/or intensity of the light source may beknown by the computer system. For example, the light source may be in aset position relative to the object and/or staging platform.

At 308, the computer system generates a virtual image of the objectusing the generated virtual light source. For example, the virtual imagemay be an approximation of the real object, generated in response toreceiving instructions from the system, and using the virtual lightsource to create a real-time estimation of an effect that the lightsource would have on the object.

At 310, the computer system may provide instructions to limit the rangeof potential visual material property values to consider. For example,the object may be adjusted to limit visual material property values froma range of 0 to 10 units. In some embodiments, the computer system mayreceive an indication of a likely material for a particular section ofthe object. In these embodiments, the computer system may identify arange of visual material property values that are possible for theindicated material. In some embodiments, the computer system may employa full range of visual material property values.

At 312, the computer system generates a grid of virtual images (e.g., aseries of virtual images of the object 106), with each point in the gridrepresenting a different combination of visual material property valuesassigned to the generated virtual image. For example, a set of visualmaterial property values may be comprised of “metalness” and“roughness.” Assuming there are ten increments of visual materialproperty values assigned to both the metalness and roughness visualmaterial properties, a 10×10 grid is created, each point in the gridassociated with a generated virtual image. To generate each of thevirtual images, the computer system may adjust the virtual image of theobject generated at 308 by adjusting the effect that the light source isestimated to have on the object in accordance with the respective visualmaterial property values. For example, if a particular virtual image tobe generated is associated with a metalness of 5 and a roughness of 3,then a highlight created using the virtual light source at 308 may beadjusted so that it approximates what a highlight would look like on anobject having a metalness of 5 and a roughness of 3. This is repeatedfor each of the fields in the grid.

At 314, the computer system provides instructions to compare eachrendered virtual image in the grid of virtual images with the capturedimages of the object. Comparison of a virtual image to an actual imagemay be performed in a number of ways. For example, the computer systemmay make a pixel-by-pixel comparison in which pixel values (e.g., red,green, and blue values) are compared for each corresponding pixel in thetwo images. A delta may then be calculated as a function of thedifference in values identified at 316. Deltas may be calculated andstored in relation to each of the virtual images in the grid of virtualimages, for example, in a grid of deltas. In some embodiments, thecomparison may include only a portion of the image that includes ahighlight on the section of the object.

In some embodiments, a delta may be calculated as an image error, (e.g.,a measure of difference between the generated image and what the imageshould have looked like). Examples of calculating an image error fromthe generated image and captured image include using a mean square erroror an absolute difference. One skilled in the art, after considering theteachings of this disclosure, would easily recognize a number of ways tocalculate these error values.

At 318, the computer system provides instructions to identify theminimum image error in the grid. In some embodiments, each field of thegrid is considered until the minimum error is identified. The computersystem then determines which visual material property values areassociated with that field. At 320, the computer system may assign thedetermined visual material property values associated with the lowesterror to the section of the object upon which the process was performed.

At 322, the computer system may determine whether additional sections ofthe object need to be processed. If the computer system determines thatadditional sections do need to be processed, then the process may thenbe repeated to approximate visual material property values for each ofthose sections. If the computer system determines that no additionalsections need to be processed, then the visual material property valuesmay be stored by the computer system with respect to each individualsection of the object.

At 324, the computer system may generate a 3D model. In generating the3D model, the computer system may render each section of the objectusing the visual material properties determined above. It should benoted that each section of the object may be assigned different visualmaterial property values. It should be noted that the identified visualmaterial property values may be independent of other materialproperties. For example, the identified visual material property valuesare independent of the object's color or opacity.

In accordance with at least some embodiments, the process 300 may beperformed by a computer system automatically (e.g., without humaninteraction). For example, a user may place an object on a stagingplatform 226 and provide various operational constraints for the 3Dimage to be captured via interface device 208. The computer system maythen perform process 300 without requiring any further interaction fromthe user.

FIG. 4 depicts a flow diagram illustrating the capture of images of theobject (e.g., the capture image step 304). In accordance with at leastsome embodiments, the process 400 may be implemented by a computersystem (e.g., the system architecture as described in FIG. 2).

In some embodiments, process 400 may begin at 402, when the computersystem receives instructions to capture images of an object. Forexample, a new object may require a full set of images captured in orderto obtain the necessary information for the computer system to generatea 3D model. In some embodiments, the user may also provide operationalconstraints, such as the number of images to be captured and/or anamount of overlap between images. The computer system may identify anangle for each rotation of a staging platform based on the number ofimages. For example, the user may indicate that six sets of imagesshould be obtained. In this example, the control unit may determine thatthe object should be repositioned in 60° increments. In other instances,the user may require additional images to capture additional detail ofthe object. For example, an array of six cameras and 15° incrementrepositioning results in at least 144 captured images of the object.

At 404, the computer system may position or reposition the object orcameras. For example, the control unit may cause one or more cameras tobe repositioned. In addition, a pan or tilt of one or more cameras maybe altered to shift a focus of that camera. In some embodiments, a levelof magnification or zoom may be updated in order to change the field ofview to the appropriate field of view for that camera.

At 408, the computer system may cause cameras to capture imageinformation associated with the object. In some embodiments, the imageinformation may include an array of images corresponding to an array ofcameras by which the image information was captured. For example, in thecaptured image information, a number of images may be taken that eachportray a different portion of the object 104 according to thedetermined field of view.

At 410, the computer system may determine whether the system needs tocapture additional images. For example, the computer system maydetermine whether images have been captured from each relevant objectposition. If the system determines that additional images are stillrequired, then the system may rotate the object and/or reposition thecameras relative to the object and return to step 304. In someembodiments, the system may capture a number of images at any angle ofrotation. The angle of rotation and/or number of rotations may bespecified in the received request. For example, the operationalcharacteristics provided by the user in the request may include anindication that the object should be rotated 12° between each set ofimages. In this example, the process described above may be repeated atotal of 30 times (for a full 360° of images). If the system determinesthat no additional images are necessary (e.g., that all of the requiredsets of images have been obtained), then the system may end the imagingprocess 400. In some embodiments, the system may generate a 3D modelfrom the captured image sets.

At 412, the computer system may reposition the object. In someembodiments, the computer system may cause the object to be repositioneda number of times in order to capture a subsequent set of images. Forexample, the computer system may identify a number of different objectorientations from which image information is to be captured. In thisexample, the computer system may, upon capturing the initial array ofimages, cause the object to be rotated, the fields of view associatedwith an array of cameras readjusted (e.g., using steps 404-408 describedabove), capture an additional array of images, and repeat this processuntil a full set of images has been obtained.

At 414, the computer system concludes obtaining image information. Insome embodiments, the captured image information may be used to generatea texture map, which may be mapped onto a shell or other object model.In some embodiments, the images in the array of images may be alignedbased on the overlapping pixels from each adjacent image in the array ofimages. In some embodiments, the images in the array of images may besaved to a database without further processing. The process 400 may becompleted upon capturing all requisite images.

In accordance with at least some embodiments, one or more of theprocesses described herein (e.g., those for identifying materialproperty values for a section of the object) may be performed withrespect to each rotation of the object. For example, material propertyvalues may be approximated for each identified section of the object ateach angle (e.g., after each rotation of the object). In someembodiments, an image difference may be calculated as a sum of all ofthe differences (or deltas) of each of the different angles.

FIG. 5 depicts an illustrative example of information that may be usedto generate a virtual light source in accordance with at least someembodiments. In FIG. 5, an object 502 may be placed within a scene (anarea to be photographed). In some embodiments, a staging object 504 mayalso be placed within the scene. The staging object may be any objecthaving a known geometry and/or specular value properties. For example,the staging object 504 may be a chrome (e.g., mirrored) ball having aminimum roughness value and a maximum metalness value. The stagingobject 504 may be placed in any position relative to the object 502,though it may be placed in such a way that it does not block capture ofan image of the object 502.

In some embodiments, a camera 506 and/or a light source 506 may have aknown position with respect to the 3D imaging system. Some embodiments,may include a depth sensor which is capable of detecting distanceinformation for the object 502 and generating a range map. The range mapmay be used to determine a geometry of the object 504. In someembodiments, an image that includes the staging object 504 may be usedto locate a highlight on the staging object 504. The location of thehighlight on the staging object 504 may then be compared to the geometryof the staging object 504 at the location of the identified highlight.In some embodiments, the system may determine an angle of the lightsource with respect to the staging object 504 and the camera 506.

Once the system has determined the geometry of the staging object 504 atthe location of the identified highlight, the system may identify alocation of a similar geometry on the object 502 (e.g., using depthinformation). In some embodiments, the system may offset the locationbased on the determined angle to compensate for a difference in sizeand/or location. A virtual light source may then be generated toapproximate the light source 508 by generating a highlight to bedepicted in a virtual image on the object 504 at the determinedlocation.

FIG. 6 depicts an illustrative example of a grid of virtual images thatmay be generated in accordance with at least some embodiments. In FIG.6, a grid 602 of virtual images may be constructed with respect to anumber of visual material properties, where each dimension of the gridis associated with a visual material property (e.g., 604 and 606).Although depicted in FIG. 6 as having two dimensions, the grid may haveany number of dimensions, each representing a different visual materialproperty. For example, in FIG. 6, the columns are depicted asrepresenting a metalness visual material property value whereas the rowsare depicted as representing a roughness visual material property value.The visual material property values may be represented in any suitableincrements.

The visual material property values of the grid 602 may be bounded byany appropriate range of values. In some embodiments, the range ofvalues may be determined based upon a type of material that the 3Dimaging system has determined it is likely that a particular section ofthe object is made of For example, the 3D imaging system may determine alikely material for a section of the object based on its color or otherproperties. In some embodiments, the 3D imaging system may receive arange of values for a particular visual material property from a uservia an interface.

In accordance with at least some embodiments, the 3D imaging system maygenerate a number of virtual images for each set of grid coordinates inthe grid of virtual images. Each image may consist of a 3D render of anobject as well as highlight that approximates what a highlight wouldlook like with the visual material property values of a grid locationusing the virtual light source. By way of illustration, the virtualimage 608 generated for grid coordinates (2, 4) may approximate what theobject would look like having a roughness value of 2 and a metalnessvalue of 4.

FIG. 7 depicts an illustrative example of a process for comparingvirtual images to an actual image and approximating visual materialproperty values based on that comparison in accordance with at leastsome embodiments. In accordance with at least some embodiments, eachgenerated virtual image 702 in a grid of virtual images (e.g., the grid602 depicted in FIG. 6) may be compared to an actual image 704 in orderto approximate the visual material property values of the objectdepicted.

In some embodiments, the 3D imaging system may identify a subsection 706of the virtual image that includes a highlight on the object. Thesubsection 706 may then be compared to a corresponding subsection 708 ofthe actual image. In particular, pixel values in the subsection 706 maybe compared to pixel values in a corresponding location within thesubsection 708. The differences in pixel values between the generatedvirtual image 706 and the pixel values in the actual image 708 may beexpressed as a delta. The delta may be calculated as a function of thepixel values (or differences in pixel values). The delta may be storedin association with the generated virtual image 706, such that eachvirtual image is associated with a delta.

Once deltas have been calculated for each of the virtual images in agrid of virtual images, the system may identify a lowest, or minimum,delta of those calculated. Once identified, the system determines thevisual material property values of the virtual image associated with thelowest identified delta. In some embodiments, a new grid of virtualimages may be generated based on the visual material property valuesassociated with the lowest delta. The specular values may then berefined by repeating the process described herein.

FIG. 8 depicts an illustrative example of a process for refining visualmaterial property values by adjusting a range of visual materialproperty values and generating a new grid using the adjusted range ofvisual material property values in accordance with at least someembodiments. In FIG. 8, a process is performed to identify a virtualimage having a lowest delta within a grid of virtual images. Asdescribed elsewhere, the object, or section of the object, may beassigned a set of approximate visual material property values. Afterapproximating these values, the system may adjust the range of valuesfor visual material properties based on the approximated values. In someembodiments, the range of values may be adjusted so that they narrow inon the approximated values. For example, the range of values for thenewly generated grid may be set so that they are bounded by values aboveand below those associated with the lowest delta.

In accordance with at least some embodiments, the range of values may bedetermined based on an amount of the lowest identified delta. In someembodiments, a minimum and maximum values in the range of values may beselected so that they vary from the approximated visual materialproperty values by an amount that is proportional to an amount of thedelta value. For example, the minimum and maximum values in the range ofvalues may be selected so that the distance from the approximated visualmaterial property values is proportional to an amount of the deltavalue. In some embodiments, the refinement process depicted in FIG. 8may be repeated a number of times to better approximate the visualmaterial property values of the object or at least a section of theobject.

FIG. 9 depicts one potential implementation of the system described inaccordance with at least some embodiments. The implementation depictedin FIG. 9 is provided for illustrative purposes and is not intended tobe limiting.

In FIG. 9 is depicted an array of cameras 902, each of which is mountedon a robotic pan/tilt platform 904. In this particular implementation,an example of suitable cameras 902 may include a number of Canon EOS 5DSR cameras which each have a 50.6 Megapixel sensor and the low-passfilter (LPF) effect cancelled in order to provide greater fine-edgesharpness and detail. Additionally, suitable robotic pan/tilt platforms904 may include any device upon which a camera 902 may be mounted whichis capable of achieving a specified position of the camera. The roboticpan/tilt platforms 904 may have a number of servomotors or otheractuators capable of positioning each camera 902 to a particularpan/tilt (e.g., by rotating the camera 902 along a horizontal orvertical axis).

Each of the cameras 902 and robotic pan/tilt platforms 904 depicted asbeing coupled with a control unit 908. Control unit 908 may be anysuitable computing device capable of performing the functions describedherein and may be an example control unit 202 as depicted in FIG. 2. Asdescribed above, the control unit 908 is in communication with aninterface device 910, which may be an example interface device 208 asdepicted in FIG. 2.

Additionally depicted is an object 912 (a vase in this example) stagedon an object positioning platform 914. The object positioning platform914 may be a turntable capable of rotating the object around a verticalaxis and may be an example object positioning platform 226 as depictedin FIG. 2. The object positioning platform 914 may also be incommunication with the control unit 908, which may provide instructionsto the object positioning platform 914 to cause it to reposition thevase 912. The positioning platform 914, and the object 912 positionedupon the positioning platform 914, may be illuminated by a light source916. In some embodiments, the light source 916 may be consist of asingle source of light in order to control lighting conditions.

In the depicted implementation, the control unit 908 may include anumber of applications configured to interact (e.g., via applicationprogramming interfaces (APIs)) to generate a 3D model of the vase 912.In particular, the control unit 908 may include at least a virtualimaging module 918 and an image comparison module 920. The virtualimaging module 918 image comparison module 920 and image comparisonmodule 920 may be an example a virtual imaging module 218 and an imagecomparison module 220 of FIG. 2, which are described in greater detailelsewhere.

The virtual imaging module 918 may be any application or combination ofapplications capable of generating a 3D model representation 924 fromimage information captured in accordance with embodiments of thedisclosure. Some suitable virtual imaging modules 918 may include aphotogrammetry software used in combination with rendering software. Forexample, the system may utilize AGISOFT PHOTOSCAN photogrammetrysoftware application or CAPTURINGREALITY photogrammetry softwareapplication, both of which are capable of performing photogrammetricprocessing of digital images and generating 3D spatial data (e.g., 3Dmodels). Additionally, the 3D models generated using the photogrammetrysoftware may be rendered using a 3D rendering software application. Someexamples of a 3D rendering software application may include game enginessuch as UNITY.

The exemplary system depicted in FIG. 9 is capable of capturing imagesof an object 912 and generating an accurate 3D model representation 924of that object. At step 1 of a process in which a 3D modelrepresentation 922 is generated from an object 912, images of the objectare captured by the cameras 902 (e.g., the Canon EOS 5DS R cameras 902in the array of cameras). The images may be captured from a number ofdifferent angles in order to obtain a complete view of the object 912.These images may then be combined by the virtual imaging module 918 inorder to generate a basic 3D model of the object (e.g., one withoutassigned visual material property values). The virtual imaging module918 then generates a series of virtual images of the vase 912. Thenumber of virtual images generated depends on the range and incrementsof the visual material properties to be assigned. For example, thevisual material properties considered may be metalness and roughness.Each metalness and roughness may consist of a range from 0 to 9,resulting in 100 generated virtual images using an increment size of one(10×10). In some embodiments, the range may be pre-defined by a user asan operational constraint. For example, the user may enter visualmaterial property values corresponding to a typical vase, or one similarto the vase 912 in which the visual material property values may bedefined to include a metalness range of 0 to 4 and a roughness rangefrom 6 to 10. In this example, if the virtual imaging module 918 uses anincrement of one, 25 virtual images would be generated.

At step 2, the image comparison module 920 calculates the image error(or delta) for each virtual image in the grid. The image error iscalculated by comparing each virtual image of the vase with the capturedimages of the vase. The image error may be calculated as a mean squareerror or as an absolute difference. Once each image error is calculated,the minimum image error is identified and selected. For example, it maybe found that a visual material property value set of {1, 2} correspondsto the minimum image error. The virtual image with 1 metalness and 2roughness will be identified as the best approximation of the actualimage.

In some embodiments, this process may be repeated a number of times atstep 3 in order to refine the visual material property values. Forexample, once a best approximation of the visual material propertyvalues has been identified from the current grid of virtual images, thevirtual imaging module 918 may generate a second grid of virtual imagesthat includes a narrower range of values as well as smaller increments.The visual material property values identified upon selection of thelowest image error from this grid will likely be more accurate thanthose first approximated. This may be repeated a number of times toimprove the accuracy of the approximation. In some embodiments, thesteps may be repeated a predetermined number of times. In someembodiments, the steps may be repeated until an image error is below apredetermined threshold value. At step 4, a final 3D model 922 may begenerated by the virtual imaging module 918 using these visual materialproperty values. The 3D model 922 may be presented on a display device924 at step 5.

Embodiments of the disclosure provide for a number of technicaladvantages over conventional 3D imaging systems. For example, a numberof conventional systems are configured to identify visual materialproperty values by generating, comparing, and adjusting images until animage error stops decreasing and begins to increase. For example, amethodology may iterate through multiple visual material propertiesuntil a low image error is found. Once a low image error is found, theiteration ends and that virtual image is selected as the most accuraterepresentation of the actual object. However, this may not be true—theremay be a different visual material property set with a more accuraterepresentation. The current system avoids these issues by generatingvirtual images for a wider range of visual material property values.

Additionally, rendering 3D models using the current system has technicaladvantages over other 3D imaging systems. For example, a number ofconventional 3D imaging systems attempt to identify a material ormaterials from which an object is made using visual material propertyvalues. Once identified, the system may assign that property to theobject, so that the object will be rendered using visual materialproperties from a material library. However, the current system rendersobjects independent of any material identification. This ensures thatthe results are accurate despite potential differences in visualmaterial property values for a material. For example, a conventional 3Dimaging system may determine that an object has visual material propertyvalues associated with brushed steel. In future renders of the object,the 3D imaging system may render the object using the brushed steel fromits material library. However, it should be noted that brushed steel canhave a range of roughness, such that one brushed steel object may havedifferent visual material properties than another brushed steel object.Accordingly, maintaining particular visual material property valuesindependent of any material may provide a more accurate render for a 3Dobject.

The various embodiments further can be implemented in a wide variety ofoperating environments, which in some cases can include one or more usercomputers, computing devices or processing devices which can be used tooperate any of a number of applications. User or client devices caninclude any of a number of general purpose personal computers, such asdesktop or laptop computers running a standard operating system, as wellas cellular, wireless, and handheld devices running mobile software andcapable of supporting a number of networking and messaging protocols.Such a system also can include a number of workstations running any of avariety of commercially-available operating systems and other knownapplications for purposes such as development and database management.These devices also can include other electronic devices, such as dummyterminals, thin-clients, gaming systems, and other devices capable ofcommunicating via a network.

Most embodiments utilize at least one network that would be familiar tothose skilled in the art for supporting communications using any of avariety of commercially-available protocols, such as TransmissionControl Protocol/Internet Protocol (“TCP/IP”), Open SystemInterconnection (“OSI”), File Transfer Protocol (“FTP”), Universal Plugand Play (“UpnP”),

Network File System (“NFS”), Common Internet File System (“CIFS”), andAppleTalk. The network can be, for example, a local area network, awide-area network, a virtual private network, the Internet, an intranet,an extranet, a public switched telephone network, an infrared network, awireless network, and any combination thereof.

In embodiments utilizing a Web server, the Web server can run any of avariety of server or mid-tier applications, including Hypertext TransferProtocol (“HTTP”) servers, FTP servers, Common Gateway Interface (“CGI”)servers, data servers, Java servers, and business application servers.The server(s) also may be capable of executing programs or scripts inresponse to requests from user devices, such as by executing one or moreWeb applications that may be implemented as one or more scripts orprograms written in any programming language, such as Java®, C, C#, orC++, or any scripting language, such as Perl, Python, or TCL, as well ascombinations thereof. The server(s) may also include database servers,including without limitation those commercially available from Oracle®,Microsoft®, Sybase®, and IBM®.

The environment can include a variety of data stores and other memoryand storage media as discussed above. These can reside in a variety oflocations, such as on a storage medium local to (and/or resident in) oneor more of the computers or remote from any or all of the computersacross the network. In a particular set of embodiments, the informationmay reside in a storage-area network (“SAN”) familiar to those skilledin the art. Similarly, any necessary files for performing the functionsattributed to the computers, servers, or other network devices may bestored locally and/or remotely, as appropriate. Where a system includescomputerized devices, each such device can include hardware elementsthat may be electrically coupled via a bus, the elements including, forexample, at least one central processing unit (“CPU”), at least oneinput device (e.g., a mouse, keyboard, controller, touch screen, orkeypad), and at least one output device (e.g., a display device,printer, or speaker). Such a system may also include one or more storagedevices, such as disk drives, optical storage devices, and solid-statestorage devices such as random access memory (“RAM”) or read-only memory(“ROM”), as well as removable media devices, memory cards, flash cards,etc.

Such devices also can include a computer-readable storage media reader,a communications device (e.g., a modem, a network card (wireless orwired)), an infrared communication device, etc.), and working memory asdescribed above. The computer-readable storage media reader can beconnected with, or configured to receive, a computer-readable storagemedium, representing remote, local, fixed, and/or removable storagedevices as well as storage media for temporarily and/or more permanentlycontaining, storing, transmitting, and retrieving computer-readableinformation. The system and various devices also typically will includea number of software applications, modules, services, or other elementslocated within at least one working memory device, including anoperating system and application programs, such as a client applicationor Web browser. It should be appreciated that alternate embodiments mayhave numerous variations from that described above. For example,customized hardware might also be used and/or particular elements mightbe implemented in hardware, software (including portable software, suchas applets), or both. Further, connection to other computing devicessuch as network input/output devices may be employed.

Storage media computer readable media for containing code, or portionsof code, can include any appropriate media known or used in the art,including storage media and communication media, such as but not limitedto volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology for storage and/or transmissionof information such as computer readable instructions, data structures,program modules, or other data, including RAM, ROM, ElectricallyErasable Programmable Read-Only Memory (“EEPROM”), flash memory or othermemory technology, Compact Disc Read-Only Memory (“CD-ROM”), digitalversatile disk (DVD), or other optical storage, magnetic cassettes,magnetic tape, magnetic disk storage, or other magnetic storage devices,or any other medium which can be used to store the desired informationand which can be accessed by a system device. Based on the disclosureand teachings provided herein, a person of ordinary skill in the artwill appreciate other ways and/or methods to implement the variousembodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the disclosure asset forth in the claims.

Other variations are within the spirit of the present disclosure. Thus,while the disclosed techniques are susceptible to various modificationsand alternative constructions, certain illustrated embodiments thereofare shown in the drawings and have been described above in detail. Itshould be understood, however, that there is no intention to limit thedisclosure to the specific form or forms disclosed, but on the contrary,the intention is to cover all modifications, alternative constructions,and equivalents falling within the spirit and scope of the disclosure,as defined in the appended claims.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless otherwise indicated herein or clearlycontradicted by context. The terms “comprising,” “having,” “including,”and “containing” are to be construed as open-ended terms (i.e., meaning“including, but not limited to,”) unless otherwise noted. The term“connected” is to be construed as partly or wholly contained within,attached to, or joined together, even if there is something intervening.Recitation of ranges of values herein are merely intended to serve as ashorthand method of referring individually to each separate valuefalling within the range, unless otherwise indicated herein and eachseparate value is incorporated into the specification as if it wereindividually recited herein. All methods described herein can beperformed in any suitable order unless otherwise indicated herein orotherwise clearly contradicted by context. The use of any and allexamples, or exemplary language (e.g., “such as”) provided herein, isintended merely to better illuminate embodiments of the disclosure anddoes not pose a limitation on the scope of the disclosure unlessotherwise claimed. No language in the specification should be construedas indicating any non-claimed element as essential to the practice ofthe disclosure.

Disjunctive language such as the phrase “at least one of X, Y, or Z,”unless specifically stated otherwise, is intended to be understoodwithin the context as used in general to present that an item, term,etc., may be either X, Y, or Z, or any combination thereof (e.g., X, Y,and/or Z). Thus, such disjunctive language is not generally intended to,and should not, imply that certain embodiments require at least one ofX, at least one of Y, or at least one of Z to each be present.

Preferred embodiments of this disclosure are described herein, includingthe best mode known to the inventors for carrying out the disclosure.Variations of those preferred embodiments may become apparent to thoseof ordinary skill in the art upon reading the foregoing description. Theinventors expect skilled artisans to employ such variations asappropriate and the inventors intend for the disclosure to be practicedotherwise than as specifically described herein. Accordingly, thisdisclosure includes all modifications and equivalents of the subjectmatter recited in the claims appended hereto as permitted by applicablelaw. Moreover, any combination of the above-described elements in allpossible variations thereof is encompassed by the disclosure unlessotherwise indicated herein or otherwise clearly contradicted by context.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to the sameextent as if each reference were individually and specifically indicatedto be incorporated by reference and were set forth in its entiretyherein.

What is claimed is:
 1. A method of generating a three-dimensional (3D)model, comprising: receiving, by a computer system, a captured image ofan object under specified lighting conditions; identifying, by thecomputer system, a section of the object based on the captured image;for at least the first section of the object, performing operations thatcomprise: generating, in accordance with the specified lightingconditions, a collection of rendered images, each of the rendered imageshaving a combination of at least X different material properties where Xis an integer greater than 1, the collection of rendered images beingrendered such that: from a first of the rendered images to an N^(th) ofthe rendered images in the collection of rendered images, where N is aninteger between one and the size of the collection of rendered images, afirst of the X different material properties remains constant whereasthe other one or more different material properties appear in varyingcombinations of values from the first of the rendered images to theN^(th) of the rendered images, from a (N+1)^(th) of the rendered imagesto a K^(th) of the rendered images, where K is an integer greater thanN, a second of said X different material properties remains constantwhereas the other one or more different material properties appear invarying combinations of values from the (N+1)^(th) of the renderedimages to the K^(th) of the rendered images, and if X is greater than 2,a (K+1)^(th) of the rendered images to a J^(th) of the rendered imagesinclude a subset of the rendered images for each of the X materialproperties other than the first and second material properties, where Jis an integer greater than K, the rendered images in each subset havinga constant value of the material property that corresponds to thatsubset and varying combinations of values for the other X-1 materialproperties, the variations in combinations of values being selected sothat there is a rendered image in the collection of rendered images foreach possible combination of values among the different materialproperties and such that each rendered image is rendered with a lightingeffect that approximates or simulates an effect of the light source onan appearance of the object with the specified combination of values forthe material properties; comparing each of the individual renderedimages of the combination of rendered images to the captured image;calculating an image error based on the comparison between eachindividual rendered image and the captured image; and assigning a visualmaterial property value associated with the individual rendered image tothe section based upon the image error.
 2. The method of claim 1,wherein the X different material properties comprise at least one ofmetalness or roughness.
 3. The method of claim 1, further comprisinggenerating a 3D model of the section using the visual material propertyvalue.
 4. The method of claim 1, wherein calculating an image errorbased on the comparison between each individual rendered image and thecaptured image comprises calculating a mean squared error in pixelvalues between each individual rendered image and the captured image. 5.The method of claim 1, wherein calculating an image error based on thecomparison between each individual rendered image and the captured imagecomprises calculating an absolute difference in pixel values betweeneach individual rendered image and the captured image.
 6. The method ofclaim 1, further comprising rotating the object and generating a secondcollection of rendered images.
 7. The method of claim 6, wherein theobject is rotated by a predetermined number of degrees.
 8. The method ofclaim 7, wherein the predetermined number of degrees is based upon anumber of angles from which collections of images should be obtained. 9.The method of claim 8, wherein the number of angles is provided by auser.
 10. The method of claim 1, wherein the method is repeated for atleast a second section of the object.
 11. A 3D imaging systemcomprising: one or more camera devices; a processor; and a memoryincluding instructions that, when executed with the processor, cause thesystem to, at least: receive a captured image of an object usingspecific lighting conditions; identify a section of the object based onthe captured image; for at least the first section of the object,perform operations that comprise: generate, based on the specificlighting conditions, a collection of rendered images, each of therendered images associated with a different combination of materialproperty values for at least X different material properties where X isan integer greater than 1, the collection of rendered images beinggenerated such that for each of X different material properties: asubset of the rendered images in the collection of rendered images isrendered such that the material property corresponding to that subsetremains constant in value whereas combinations of values for the X-1other different material properties vary among the rendered images inthe that subset; compare each of the rendered images of the collectionof rendered images to the captured image; calculate an image error foreach rendered image based on the comparison between each rendered imageand the captured image; and assign to the first section a combination ofmaterial property values associated with one of the rendered imagesbased upon the image error.
 12. The 3D imaging system of claim 11,wherein the first section is assigned the one of the rendered imagesassociated with the lowest image error.
 13. The 3D imaging system ofclaim 11, wherein the collection of rendered images is generated suchthat rendered images generated from each possible combination ofmaterial property values for the X different material properties areincluded in the collection of rendered images.
 14. The 3D imaging systemof claim 11, wherein each material property value of the X differentmaterial properties is associated with a range of values.
 15. The 3Dimaging system of claim 14, wherein the range of values for eachmaterial property value of the X different material properties isdifferent.
 16. The 3D imaging system of claim 14, wherein the range ofvalues is predetermined based on a material from which the section isassumed to be made from.
 17. The 3D imaging system of claim 11, whereinthe specific lighting conditions comprises a single light source at aknown location.
 18. The 3D imaging system of claim 11, wherein the Xdifferent material properties comprise properties of the section thatimpact the section's appearance.
 19. The 3D imaging system of claim 11,further comprising a rotating platform, wherein the rotating platform isconfigured to rotate the object.
 20. The 3D imaging system of claim 19,wherein the instructions further cause the system to obtain a pluralityof collections of images from different angles.